Deep learning-based full-reference image quality assessment (FR-IQA) models typically rely on the feature distance between the reference and distorted images. However, the underlying assumption of these models that the distance in the deep feature domain could quantify the quality degradation does not scientifically align with the invariant texture perception, especially when the images are generated artificially by neural networks. In this paper, we bring a radical shift in inferring the quality with learned features and propose the Deep Image Dependency (DID) based FR-IQA model. The feature dependency facilitates the comparisons of deep learning features in a high-order manner with Brownian distance covariance, which is characterized by the joint distribution of the features from reference and test images, as well as their marginal distributions. This enables the quantification of the feature dependency against nonlinear transformation, which is far beyond the computation of the numerical errors in the feature space. Experiments on image quality prediction, texture image similarity, and geometric invariance validate the superior performance of our proposed measure.
translated by 谷歌翻译
自然图像的统计规律(称为自然场景统计数据)在不引用图像质量评估中起重要作用。但是,人们普遍认为,通常是计算机生成的屏幕内容图像(SCI)不持有此类统计信息。在这里,我们首次尝试学习SCI的统计数据,基于可以有效确定SCI的质量。所提出的方法的基本机制是基于一个狂野的假设,即没有物理上获得的SCI仍然遵守某些可以以学习方式理解的统计数据。我们从经验上表明,在质量评估中可以有效利用统计偏差,并且在不同的环境中进行评估时,提出的方法优越。广泛的实验结果表明,与现有的NR-IQA模型相比,基于深度统计的SCI质量评估(DFSS-IQA)模型可提供有希望的性能,并在跨数据库设置中显示出很高的概括能力。我们的方法的实现可在https://github.com/baoliang93/dfss-iqa上公开获得。
translated by 谷歌翻译
现有的基于深度学习的全参考IQA(FR-IQA)模型通常通过明确比较特征,以确定性的方式预测图像质量,从而衡量图像严重扭曲的图像是多远,相应的功能与参考的空间相对远。图片。本文中,我们从不同的角度看这个问题,并提议从统计分布的角度对知觉空间中的质量降解进行建模。因此,根据深度特征域中的Wasserstein距离来测量质量。更具体地说,根据执行最终质量评分,测量了预训练VGG网络的每个阶段的1Dwasserstein距离。 Deep Wasserstein距离(DEEPWSD)在神经网络的功能上执行的,可以更好地解释由各种扭曲引起的质量污染,并提出了高级质量预测能力。广泛的实验和理论分析表明,在质量预测和优化方面,提出的DEEPWSD的优越性。
translated by 谷歌翻译
在本文中,我们提出了通过特征级伪参考(PR)幻觉提出的无引用(NR)图像质量评估(IQA)方法。提出的质量评估框架基于自然图像统计行为的先前模型,并植根于以下观点,即可以很好地利用具有感知意义的特征来表征视觉质量。本文中,通过以原始参考为监督的相互学习方案学习了扭曲的图像中的PR特征,并通过三重态约束进一步确保PR特征的区分特性。给定质量推断的扭曲图像,特征水平的分离是用可逆神经层进行最终质量预测的,导致PR和相应的失真特征以进行比较。在四个流行的IQA数据库中证明了我们提出的方法的有效性,跨数据库评估的卓越性能也揭示了我们方法的高概括能力。我们的方法的实现可在https://github.com/baoliang93/fpr上公开获得。
translated by 谷歌翻译
KDD CUP 2022提出了有关空间动态风能数据集的时间序列预测任务,其中要求参与者预测未来一代给定历史上下文因素。评估指标包含RMSE和MAE。本文介绍了团队88VIP的解决方案,该解决方案主要包括两种模型:梯度增强决策树,以记住基本数据模式和复发性神经网络,以捕获深层和潜在的概率过渡。结合这些模型有助于应对风力的波动,以及训练子模型对预测的异质时间尺度(从几分钟到几天)的杰出特性的目标。此外,还详细介绍了功能工程,插补技术和离线评估的设计。拟议的解决方案在第3阶段的总体在线得分为-45.213。
translated by 谷歌翻译
修剪是卷积神经网络(CNNS)模型压缩的有效技术,但是由于较大的设计空间,很难找到最佳的修剪政策。为了提高修剪的可用性,已经开发了许多自动修剪方法。最近,由于其坚实的理论基础和高采样效率,贝叶斯优化(BO)被认为是一种自动修剪的竞争算法。但是,BO受到维度的诅咒。由于设计空间的尺寸增加,在修剪深CNN时,BO的性能会恶化。我们提出了一种新颖的聚类算法,该算法降低了设计空间的尺寸以加快搜索过程。随后,提出了回滚算法以恢复高维设计空间,以便获得更高的修剪精度。我们验证了有关Resnet,MobilenetV1和MobilenetV2模型的建议方法。实验表明,提出的方法在修剪深CNN而不会增加运行时间时显着提高BO的收敛速率。源代码可在https://github.com/fanhanwei/bocr上获得。
translated by 谷歌翻译
Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samples are not reliable for ignoring important clustering information. To solve these problems, we propose a Cluster-guided Contrastive deep Graph Clustering network (CCGC) by mining the intrinsic supervision information in the high-confidence clustering results. Specifically, instead of conducting complex node or edge perturbation, we construct two views of the graph by designing special Siamese encoders whose weights are not shared between the sibling sub-networks. Then, guided by the high-confidence clustering information, we carefully select and construct the positive samples from the same high-confidence cluster in two views. Moreover, to construct semantic meaningful negative sample pairs, we regard the centers of different high-confidence clusters as negative samples, thus improving the discriminative capability and reliability of the constructed sample pairs. Lastly, we design an objective function to pull close the samples from the same cluster while pushing away those from other clusters by maximizing and minimizing the cross-view cosine similarity between positive and negative samples. Extensive experimental results on six datasets demonstrate the effectiveness of CCGC compared with the existing state-of-the-art algorithms.
translated by 谷歌翻译
To generate high quality rendering images for real time applications, it is often to trace only a few samples-per-pixel (spp) at a lower resolution and then supersample to the high resolution. Based on the observation that the rendered pixels at a low resolution are typically highly aliased, we present a novel method for neural supersampling based on ray tracing 1/4-spp samples at the high resolution. Our key insight is that the ray-traced samples at the target resolution are accurate and reliable, which makes the supersampling an interpolation problem. We present a mask-reinforced neural network to reconstruct and interpolate high-quality image sequences. First, a novel temporal accumulation network is introduced to compute the correlation between current and previous features to significantly improve their temporal stability. Then a reconstruct network based on a multi-scale U-Net with skip connections is adopted for reconstruction and generation of the desired high-resolution image. Experimental results and comparisons have shown that our proposed method can generate higher quality results of supersampling, without increasing the total number of ray-tracing samples, over current state-of-the-art methods.
translated by 谷歌翻译
Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.
translated by 谷歌翻译
Representing and synthesizing novel views in real-world dynamic scenes from casual monocular videos is a long-standing problem. Existing solutions typically approach dynamic scenes by applying geometry techniques or utilizing temporal information between several adjacent frames without considering the underlying background distribution in the entire scene or the transmittance over the ray dimension, limiting their performance on static and occlusion areas. Our approach $\textbf{D}$istribution-$\textbf{D}$riven neural radiance fields offers high-quality view synthesis and a 3D solution to $\textbf{D}$etach the background from the entire $\textbf{D}$ynamic scene, which is called $\text{D}^4$NeRF. Specifically, it employs a neural representation to capture the scene distribution in the static background and a 6D-input NeRF to represent dynamic objects, respectively. Each ray sample is given an additional occlusion weight to indicate the transmittance lying in the static and dynamic components. We evaluate $\text{D}^4$NeRF on public dynamic scenes and our urban driving scenes acquired from an autonomous-driving dataset. Extensive experiments demonstrate that our approach outperforms previous methods in rendering texture details and motion areas while also producing a clean static background. Our code will be released at https://github.com/Luciferbobo/D4NeRF.
translated by 谷歌翻译